{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fisher's Linear Discriminant"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "np.set_printoptions(suppress=True)\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn import datasets\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since it is largely geometric, the Linear Discriminant won't look like other methods we've seen (no gradients!). First we take the class means and calculate $\\bSigma_w$ as described in the {doc}`concept section </content/c3/s2/fisher_discriminant>`. Estimating $\\bbetahat$ is then as simple as calculating $\\bSigma_w^{-1}(\\bmu_1 - \\bmu_0)$. Let's demonstrate the model using the {doc}`breast cancer </content/appendix/data>` dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import data\n",
    "cancer = datasets.load_breast_cancer()\n",
    "X = cancer['data']\n",
    "y = cancer['target']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{note}\n",
    "Note that the value we return for each observation, $f(\\bx_n)$, is not a fitted value. Instead, we look at the distribution of the $f(\\bx_n)$ by class and determine a classification rule based on this distribution. For instance, we might predict observation $n$ came from class 1 if $f(\\bx_n) \\geq c$ (for some constant c) and from class 0 otherwise. \n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class FisherLinearDiscriminant:\n",
    "    \n",
    "    def fit(self, X, y):\n",
    "        ## Save stuff\n",
    "        self.X = X\n",
    "        self.y = y\n",
    "        self.N, self.D = self.X.shape\n",
    "        \n",
    "        ## Calculate class means\n",
    "        X0 = X[y == 0]\n",
    "        X1 = X[y == 1]\n",
    "        mu0 = X0.mean(0)\n",
    "        mu1 = X1.mean(0)\n",
    "        \n",
    "        ## Sigma_w\n",
    "        Sigma_w = np.empty((self.D, self.D))\n",
    "        for x0 in X0:\n",
    "            x0_minus_mu0 = (x0 - mu0).reshape(-1, 1)\n",
    "            Sigma_w += np.dot(x0_minus_mu0, x0_minus_mu0.T)\n",
    "        for x1 in X1:\n",
    "            x1_minus_mu1 = (x1 - mu1).reshape(-1, 1)\n",
    "            Sigma_w += np.dot(x1_minus_mu1, x1_minus_mu1.T)            \n",
    "        Sigma_w_inverse = np.linalg.inv(Sigma_w)\n",
    "        \n",
    "        ## Beta\n",
    "        self.beta = np.dot(Sigma_w_inverse, mu1 - mu0)\n",
    "        self.f = np.dot(X, self.beta)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now fit the model on the {doc}`breast cancer </content/appendix/data>` dataset, as below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = FisherLinearDiscriminant()\n",
    "model.fit(X, y);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have fit the model, we can look at the distribution of $f(\\bx_n)$ by class. We hope to see a significant separation between classes and a significant clustering within classes. The histogram below shows that we've nearly separated the two classes and the two classes are decently clustered. We would presumably choose a cutoff somewhere between $f(x) = -.09$ and $f(x) = -.08$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbcAAAFYCAYAAADQhbK8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5xdVX338c+PJDoUsSQhYCSEuwoEE2JErSViEEixRShSoS0GRcYbWlr7PFptH+OlRasFffp4C3esCoIiYhWlFAyKVQJNIAEFxKgJ4ZIIiOViLr/nj70Hh2FmzplzmZksPu/X67xmzt5r773WOWfme9bea+8dmYkkSSXZZqwrIElSpxlukqTiGG6SpOIYbpKk4hhukqTiGG6SpOIYbpKk4hhukqTiGG6FiIiTIiIjYu9B5k2s5y3uN21xRDR9Bn9EHB0Rf9Oh6hYrIt4YEXdExG8j4sEmyu8SEV+NiPsjYktEnDFEuX+NiCs6XNe/joibI6Lh/4G+z0tETOxkHYbY1ssi4ssRcXf9Om6IiKsiYlFETOhfn27XRVsvw+3p62zgZSMofzRguA0jIp4LLAGuBxYAr2pisfOB5wOLgD8A/nmQ9e4FvBn4QKfqWvsssFO97XEhIk4Dvg9MAd5N9Rq+Ebgd+Azwx2NXO21Nuv4tTONTZq4B1ox1PZoVEc/MzMfHuh4N7ANMAC7IzO81KhwRzwEOBU7KzG8OU/Q0YEVmLutMNSuZ+WhEXAj8LXBeJ9fdioiYD5wB/L/MfOeA2ZfXvdrtRr9m2hrZc3uaGrhbJyKeFxGXRcR9EfFYRPwiIi6pd2meT/Xtfpd611RGxOoB61sYET+IiEcj4qGI+FpEPH9AmRMi4sf1+m+JiKMi4tqIuHawukXErIj4dkT8BvhyROwdEZ+PiJ/V27krIj4TEZOHWP4F9fL/U7fnDfX8E+t6/CYirql7Rs28ZkO2sX6N+tpxdb3984dZ11eBdUAAF9TlPzhIuWcCfwl8ccD0vSNiY0R8YMD0z0TEwxExr5k2ARcB+0XEHzRZft/6NXskItZFxAf7dmtGxGvrdswepB3XRsQPGqz7PcCvgP892MzM/Glm3jzUws18Pob7nI+kjMY/36zyTBjkj3BCE8t9A3gQeCuwHtgFOJLqC9CHgGnAi4Gj6vJP9KIiYiHw78B/Aq8DngV8EPheRMzJzLURcRjwBeDrwLuAHYFPAD1Uu5wGczlwDvBRYAvwXKre5mnAA8CewHuBbzL4LtZLgLOAjwNvA86NiH2AQ6j+kU4CPkkVHC8Z7sVp1Mb6NboR+L/A24GbgPuHWeU/ABuA44HD6ml3DVLupcAOwHX9J2bmnRFxNvDXEfGvmbk+Iv4P1S68V4+gl7cc+DWwkGp3aiNfA84FTgeOqNuxBVhcz7ubahfq2/oWqL8AvAJ4w1ArrY+lHQJ8LTMfa7LuAzXz+Rjuc84Iymi8y0wfBTyAk4Bs8Fjcr/zi6u1PqIImgaOGWf/5wJoh5i0D7gAm9pu2B7AROKN+fj2wEoh+ZebW2712wPoW19P/qkGbJwJ/WJc9cJDlX99v2mRgE1WgPLvf9HfWZXdrsK1m2viqel2HNPmeXQJ8v0GZd1OFxzMGmfcc4H+AjwEnA5uBP2vhs3Md8J0GZfpe0/cMmH4W8DCwQ79yDwHb9StzBlXYbDvM+neu1396k3V+4vPb7Oejyc95wzI+to6H30TKcwxVD6v/46UNltlA1Wv4SEScUvdumhIR21GF1MWZualvemb+jGpgwCvqb+XzgK9k/R+kLnMT8LNhVn/ZgG09IyLeW+9SfJQqWPp6NM9/ytLwrX7begC4D/ivzPx1vzI/rn/u2k4bh2nDcOYA/92gzHOBX2fmbwfOyMx7qHq/7wA+B7wzM7/cQj3ur7fTjIHrv4iqFzurfr4E+D3gBICI6KHapX1hZj7aQt2a1sTno5nPect/CxpfDLfyrMzMZf0fVLvLhlQHzmFUvZPTgdvr4xVvbWJ7k6mOG60bZN49VKPedqTaBXjfIGXuHWbdA9d5OtU39n8DXg0cBPxpPa9nkOUfGPD8t0NMG2r5Ps20cUQiYntgLxqHWw/9dgEP4g7gmcAPMvNTI61H7VFg2ybLDny/+p7vApCZd1PtTn5LPf04qtfncw3Wu6Gux25N1mMww34+mvmct/m3oHHEY24CIDPvAl4fEQHMBk4FPh0RqzPzW8Ms+gDVbpznDDLvOVT/tNZTfYveaZAyOwO/GKpaA54fT9UD+HDfhIh41jB165Rm2jhSc6gCs1G4baAK16eIiAVUofED4OURMTszV/Sb/09UXyx2pNottxZYmJm/GbCqKVTvUTN25snHBneuf67tN+3TVINqXkR1/O26zLx1uJVm5qZ6YNFh0frI2Iafj2Y+5238LWgcseemJ8nKcn53Tlvf7qbHGeTbfWb+D1XP8Lh69yMAEbEb1Xlb383MzVTfhI+t/2H0lXkR1XGrZv0eVUj2N+QghU5ppo0trPZAqrasalDux8CkiJjRf2JEzKUawHE21UCMXwD/NGDZucB0qtGWe1IF9PxBtrEH8JMm6/1nA54fD/yG6ngqAJn5n8BtVMfaXk51Pl0zPgJMpTqG+BQRsUdEvHCY5Zv+fAzzOR9RGY1f9txE/Q/jk8DFwJ1UoytPohqA8Z91sVuBKfXumWXAY5l5Sz3vH6hGEn4jIj5NdQzmA1QDC/6lLvN+4DvAZRGxhKo3sZhqt96WJqt6JbAoIm6p6/mnVOEyGppp40gcCNzaRA9laf3zIOrzEqO6Cs23qF7Pd2TmlvqUgHMjYn5m9i0zF3hFZj5SL/cMqqH2T4iIHYDnUY0obcYp9dD/G6hGS76JaqDSwKuxfJbqM7Ue+EozK87MpVFdBeeMiNiXahDTL6h6rofW2/pzYKjTAYb9fDTzOW/yb0Fbg7Ee0eKjMw9+N1py70HmTWT40ZI7ARdQDcl/hOof4HeBI/qV3w74Er/bRbd6wDYWUu0ee5TqH/7lwPMHlPlzqh7C41Q9lmOodstdNqDc4nobEwdM35FqAMMD9eMLVANmkupE6EbLrwb+bcC0Q+qyr2riNR62jYxgtCTVEPzzm3xvfwicV//+HKrdgtcCz+xXZgJVb+n6+vlM4P5+8ydR9bC2HbDuvwAeA6Y2qEPfazoLuKZ+De6hOgVim0HKT6/Lf6yFz/IfUI0kXUfVE/sVVZD/Zd+2GGS0ZKPPB819zhuW8bF1PKJ+Q6VRV+9quxP4x8z80FjXZ7yKiJOoehPTs+6FNbHM0UBvZh5ZP59LFaYvHFDuW8D6zDyxw3U+hep44PMy885OrltqhsfcNCoiYtv6ahHHRsQrorpayFVU347PHuPqjXefpxqw8bZGBft5EdXu4z7zBjynPvn8lXTwmpURsV9E/Em9zq8ZbBor9tw0KurjPRdTnXM3lerk4+uA92bmyuGWFUTES4G5mfnpDq5zITA5M7/UwXVeS7Vb8Xrgz7M6NUAadYabJKk47paUJBXHcJMkFWdUz3NbuHBhXnnllaO5SUnS+BeNi4zMqPbc1q9v9go/kiS1zt2SkqTiGG6SpOIYbpKk4njhZEnqoI0bN7JmzRoee+yxsa7KuNPT08OMGTOYNGlS17dluElSB61Zs4btt9+e3XffnX53eHray0w2bNjAmjVr2GOPkdzpqjXulpSkDnrssceYOnWqwTZARDB16tRR69EabpLUYQbb4EbzdTHcJKkw99xzD8cffzx77bUX++23H0ceeSS33347q1evZtas7txQ/PHHH+d1r3sde++9Ny95yUtYvXp1V7bTLI+5SVIXLV3V2d1w8/fvGXZ+ZnLMMcewaNEiLrroIgCWL1/Ovffey6677trRuvR3zjnnMHnyZO68804uuugi3v3ud3PxxRd3bXuN2HOTpIJcc801TJo0ibe85S1PTJszZw4HH3zwk8qtXr2agw8+mLlz5zJ37lyuv/56ANatW8f8+fOZM2cOs2bN4rrrrmPz5s2cdNJJzJo1iwMOOIAzzzzzKdu9/PLLWbRoEQCvfe1rufrqqxnLu87Yc5OkgqxcuZIXvehFDcvttNNOXHXVVfT09HDHHXdwwgknsGzZMr74xS9yxBFH8L73vY/NmzfzyCOPsHz5ctauXcvKldWtFx988MGnrG/t2rVP9AwnTpzI7//+77NhwwZ23HHHzjawSYabpGItWbJiyHm9vbNHsSbjz8aNGzn11FNZvnw5EyZM4PbbbwfgxS9+MW984xvZuHEjRx99NHPmzGHPPffkrrvu4h3veAevfvWrOfzww5+yvsF6aWM5sMbdkpJUkP33358bb7yxYbkzzzyTnXfemRUrVrBs2TJ++9vfAjB//nyWLl3KLrvswoknnsiFF17I5MmTWbFiBYcccgif+tSneNOb3vSU9c2YMYNf/vKXAGzatImHHnqIKVOmdLZxI2C4SVJBFixYwOOPP85ZZ531xLQbbriB7373u08q99BDDzF9+nS22WYbPv/5z7N582YAfv7zn7PTTjtxyimncPLJJ3PTTTexfv16tmzZwrHHHsuHPvQhbrrppqds96ijjuKCCy4A4NJLL2XBggVj2nNzt6QkFSQiuOyyyzjttNP4yEc+Qk9PD7vvvjuf+MQnnlTubW97G8ceeyyXXHIJr3zlK9luu+0AuPbaa/nYxz7GpEmTeNaznsWFF17I2rVrecMb3sCWLVsAOP3005+y3ZNPPpkTTzyRvffemylTpjwxUnOsxGiOZpk3b14uW7Zs1LYn6eltLI653Xbbbey7775dWXcJhnh9tu6blUqSNBoMN0lScQw3SVJxDDdJUnEMN0lScRqGW0T0RMSPImJFRKyKiA/U08+PiJ9FxPL6Maf71ZUkqbFmem6PAwsyczYwB1gYES+t5/2vzJxTP5Z3rZaSpKaNxS1vli5dyty5c5k4cSKXXnppV7YxEg1P4s7qRLjf1E8n1Y+xu9SzJG1FhjvXrhWNzs8bq1vezJw5k/PPP5+Pf/zjXdvGSDR1zC0iJkTEcuA+4KrM/GE96x8j4uaIODMintm1WkqSmjJWt7zZfffdeeELX8g224yPoRxNXX4rMzcDcyJiB+CyiJgF/B1wD/AMYAnwbuCDA5eNiF6gF6pkl6SR8Mr+IzNWt7wZb0YUsZn5IHAtsDAz12XlceA84KAhllmSmfMyc960adParrAkqX0bN27klFNO4YADDuC4447j1ltvBapb3px33nksXryYW265he233/5Jt7y58sorefaznz3GtW+smdGS0+oeGxGxLfAq4McRMb2eFsDRwMpuVlSS1NhY3fJmvGmm5zYduCYibgZuoDrm9g3gCxFxC3ALsCPw4e5VU5LUjLG65c1408xoyZuBAweZvqArNZIktWysbnlzww03cMwxx/DAAw9wxRVX8P73v59Vq1Z1v8FD8JY3ksa1dgaUeMub8cdb3kiS1CLDTZJUHMNNklQcw02SOmw0xzJsTUbzdTHcJKmDenp62LBhgwE3QGayYcMGenp6RmV7TV1+S5LUnBkzZrBmzRruv//+sa7KuNPT08OMGTNGZVuGmyR10KRJk9hjjz3GuhpPe+6WlCQVx56bpDHX6XueSfbcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnFaRhuEdETET+KiBURsSoiPlBP3yMifhgRd0TExRHxjO5XV5KkxprpuT0OLMjM2cAcYGFEvBT4KHBmZu4DPACc3L1qSpLUvIbhlpXf1E8n1Y8EFgCX1tMvAI7uSg0lSRqhpo65RcSEiFgO3AdcBfwUeDAzN9VF1gC7dKeKkiSNTFPhlpmbM3MOMAM4CNh3sGKDLRsRvRGxLCKW3X///a3XVJKkJo1otGRmPghcC7wU2CEiJtazZgB3D7HMksycl5nzpk2b1k5dJUlqSjOjJadFxA7179sCrwJuA64BXlsXWwRc3q1KSpI0EhMbF2E6cEFETKAKwy9n5jci4lbgooj4MPDfwDldrKckSU1rGG6ZeTNw4CDT76I6/iZJ0rjiFUokScUx3CRJxWnmmJskjUtLlqwYk/X29s7uynbVOfbcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnF8a4Akp6WunVHAY0P9twkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnFaRhuEbFrRFwTEbdFxKqI+Kt6+uKIWBsRy+vHkd2vriRJjTVz+a1NwLsy86aI2B64MSKuquedmZkf7171JEkauYbhlpnrgHX17w9HxG3ALt2umCRJrRrRhZMjYnfgQOCHwMuBUyPi9cAyqt7dA4Ms0wv0AsycObPN6krS+DbcBZl7e2ePYk2e3poeUBIRzwK+ApyWmb8GPgPsBcyh6tn9y2DLZeaSzJyXmfOmTZvWgSpLkjS8psItIiZRBdsXMvOrAJl5b2ZuzswtwFnAQd2rpiRJzWtmtGQA5wC3ZeYZ/aZP71fsGGBl56snSdLINXPM7eXAicAtEbG8nvZe4ISImAMksBp4c1dqKEnSCDUzWvJ7QAwy65udr44kSe3zCiWSpOKM6FQASU9vDnPX1sKemySpOIabJKk4hpskqTiGmySpOIabJKk4hpskqTieCqC2LF31WFPl5u/f0+WaSNLv2HOTJBXHcJMkFcdwkyQVx3CTJBXHcJMkFcdwkyQVx1MBJHXdcHcTkLrBnpskqTiGmySpOIabJKk4hpskqTiGmySpOIabJKk4ngogqSMc7q/xxJ6bJKk4hpskqTiGmySpOA3DLSJ2jYhrIuK2iFgVEX9VT58SEVdFxB31z8ndr64kSY0103PbBLwrM/cFXgq8PSL2A94DXJ2Z+wBX188lSRpzDcMtM9dl5k317w8DtwG7AK8BLqiLXQAc3a1KSpI0EiM6FSAidgcOBH4I7JyZ66AKwIjYaYhleoFegJkzZ7ZTV0kaFzztYfxrekBJRDwL+ApwWmb+utnlMnNJZs7LzHnTpk1rpY6SJI1IU+EWEZOogu0LmfnVevK9ETG9nj8duK87VZQkaWSaGS0ZwDnAbZl5Rr9ZXwcW1b8vAi7vfPUkSRq5Zo65vRw4EbglIpbX094LfAT4ckScDPwCOK47VZQkaWQahltmfg+IIWYf2tnqSJLUPq9QIkkqjuEmSSqO4SZJKo7hJkkqjuEmSSqO4SZJKo7hJkkqjuEmSSrOiO4KIG1Nlq56rKly8/fv6XJNJI02e26SpOIYbpKk4hhukqTiGG6SpOIYbpKk4hhukqTieCqAxpVmh+9L0nDsuUmSimO4SZKKY7hJkopjuEmSimO4SZKKY7hJkopjuEmSimO4SZKKY7hJkopjuEmSitMw3CLi3Ii4LyJW9pu2OCLWRsTy+nFkd6spSVLzmum5nQ8sHGT6mZk5p358s7PVkiSpdQ3DLTOXAr8ahbpIktQR7dwV4NSIeD2wDHhXZj4wWKGI6AV6AWbOnNnG5qSx1ewdC+bv39PlmkhqpNUBJZ8B9gLmAOuAfxmqYGYuycx5mTlv2rRpLW5OkqTmtRRumXlvZm7OzC3AWcBBna2WJEmtayncImJ6v6fHACuHKitJ0mhreMwtIr4EHALsGBFrgPcDh0TEHCCB1cCbu1hHSZJGpGG4ZeYJg0w+pwt1kSSpI7xCiSSpOO2cCqBxxGHqkvQ79twkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnF8VQAPe01exqFpK2HPTdJUnEMN0lScQw3SVJxDDdJUnEMN0lScQw3SVJxPBVAg3J4/NPTkiUrxroKRWv0+vb2zh6lmpTPnpskqTiGmySpOIabJKk4hpskqTiGmySpOIabJKk4ngowznV6SL5D/Luv2dd4/v49Xa6J9PRlz02SVBzDTZJUHMNNklSchuEWEedGxH0RsbLftCkRcVVE3FH/nNzdakqS1Lxmem7nAwsHTHsPcHVm7gNcXT+XJGlcaBhumbkU+NWAya8BLqh/vwA4usP1kiSpZa2eCrBzZq4DyMx1EbHTUAUjohfoBZg5c2aLm9PWzlMQJI2mrg8oycwlmTkvM+dNmzat25uTJKnlcLs3IqYD1D/v61yVJElqT6vh9nVgUf37IuDyzlRHkqT2NXMqwJeAHwDPj4g1EXEy8BHgsIi4Azisfi5J0rjQcEBJZp4wxKxDO1wXSZI6wiuUSJKK410BpDEyVncPWLJkRUfXJ41H9twkScUx3CRJxTHcJEnFMdwkScUx3CRJxTHcJEnF8VQASRonhjtNo7d39ijWZOtnz02SVBzDTZJUHMNNklQcw02SVBzDTZJUHMNNklQcTwWQCtHsXQakpwN7bpKk4hhukqTiGG6SpOIYbpKk4hhukqTiGG6SpOJ4KsAYcdi2JHWPPTdJUnEMN0lScQw3SVJx2jrmFhGrgYeBzcCmzJzXiUpJktSOTgwoeWVmru/AeiRJ6gh3S0qSitNuzy2B70REAp/LzCUDC0REL9ALMHPmzDY3J6ldt9+9salyz3vupC7XROqedntuL8/MucAfAW+PiPkDC2Tmksycl5nzpk2b1ubmJElqrK1wy8y765/3AZcBB3WiUpIktaPlcIuI7SJi+77fgcOBlZ2qmCRJrWrnmNvOwGUR0beeL2bmlR2plSRJbWg53DLzLmB2B+siSVJHeCqAJKk43hVAGue8g4Q0cvbcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxfFUgA5z2LakUixZsmLY+b294/c6HvbcJEnFMdwkScUx3CRJxTHcJEnFMdwkScUx3CRJxfFUAKlAX79k5VhXQR3WaFj+cMbzkP1usecmSSqO4SZJKo7hJkkqjuEmSSqO4SZJKo7hJkkqzlZ5KsBYXHl//v49o75Nbf0aDck/6rhZLS/bbbffvXFMtvu8504ak+2WrJ3TCLZW9twkScUx3CRJxTHcJEnFaSvcImJhRPwkIu6MiPd0qlKSJLWj5XCLiAnAp4A/AvYDToiI/TpVMUmSWtVOz+0g4M7MvCszfwtcBLymM9WSJKl1kZmtLRjxWmBhZr6pfn4i8JLMPHVAuV6gt376fOAnLdZ1R2B9i8uOVyW2CWzX1qTENoHt2prsCPw4Mxd2cqXtnOcWg0x7SlJm5hJgSRvbqTYWsSwz57W7nvGkxDaB7dqalNgmsF1bk7pNHQ02aG+35Bpg137PZwB3t1cdSZLa10643QDsExF7RMQzgOOBr3emWpIkta7l3ZKZuSkiTgW+DUwAzs3MVR2r2VO1vWtzHCqxTWC7tiYltgls19akK21qeUCJJEnjlVcokSQVx3CTJBVn3IRbREyJiKsi4o765+Qhyl0ZEQ9GxDcGTP9CfSmwlRFxbkSMi/tmdKBdp9aXN8uI2HF0at1YB9q1R0T8sF7+4npQ0pgaQZsW1WXuiIhF/aa/LiJujohVEfHPo1fz4XWgXSdExC11264cL5/DdtoVEdtHxPJ+j/UR8YnRbcGgdW33vXpGRCyJiNsj4scRcezo1X5oHWjXtfX/9773a6eGG83McfEA/hl4T/37e4CPDlHuUOBPgG8MmH4k1bl3AXwJeOtYt6lD7ToQ2B1YDew41u3pYLu+DBxf//7Z8fB+NdMmYApwV/1zcv37ZGAq8AtgWl3uAuDQsW5TB9o1Ebiv77NXr2vxWLep3XYNUu5GYP7W3ibgA8CH69+3GS//MzrQrmuBeSPa5lg3ul/DfgJMr3+fDvxkmLKHDPxnOWD+XwP/ONZt6mS7GH/h1nK7qL6ArAcm1s9fBnx7a2gTcALwuX7PP1dPezHwH/2mnwh8eqzb1IF2TQLuB3ar37fPAr1j3aZ22zWgzD7AL6kH2G3Nbarbsd1Yt6ML7RpxuI2b3ZLAzpm5DqD+2bjbOYh6d+SJwJUdrFs7OtKucaiddk0FHszMTfXzNcAuHa5fK5pp0y5U/0D69NX9TuAFEbF7REwEjubJFzkYSy23KzM3Am8FbqG6SMN+wDndrW7T2nm/+jsBuDjr/6JjrOU2RcQO9fMPRcRNEXFJROzc3eo2rRPv1Xn1Lsl/iIjBrpD1JO1cfmvEIuI/gOcMMut9HdzMp4GlmXldB9c5rFFq16jrYruaunRbN3SgTYPWPTMfiIi3AhcDW4DrgT1bq+XIdatd9ZfFt1LtHr8L+Ffg74APt1LPkepWuwY8P57qC/Go6GKbJlJdKer7mfk3EfE3wMcZpbZ1+b36i8xcGxHbA1+hatOFw61sVMMtM1811LyIuDcipmfmuoiYTrWff0Qi4v3ANODNbVRzxLrdrrHSxXatB3aIiIl1723ULt3WgTatodrN2mcG1S4TMvMK4Ip6Xb3A5g5Vu6EutmtOvf6f1uv6MtUxk1HRzferXsdsqt3jN3amxo11sU0bgEeAy+rplwAnd6LOzejy39ba+ufDEfFFqrvSDBtu42m35NeBvtExi4DLR7JwRLwJOIJqH+2WDtetHW21axxruV317p9rgNe2snwXNdOmbwOHR8TkesTX4fU0+kZw1dPfBpzd9Ro3p512rQX2i4hpdbnDgNu6XN9mtfV+1U6gGoA2XrTcpvrv6gp+FxCHArd2t7pNa7ldETGxb4RuvSfhj4GVDbc41gca+x08nApcDdxR/5xST58HnN2v3HVUB7gfpUr6I+rpm4CfAsvrx/8Z6zZ1qF3vrJ9vourdnD3abehSu/YEfkR1rOoS4JlbUZveWNf7TuAN/aZ/ieqfya3UI0HHw6MD7XoLVaDdTPXPc+pYt6kT7arn3QW8YKzb0sH3ajdgaf1eXQ3MHOs2tdsuYDuq0aw3A6uATwITGm3Ty29JkooznnZLSpLUEYabJKk4hpskqTiGmySpOIabJKk4hpskqTiGm9SmiDglIu6KiE0R8bkB8ybXV2fYqwPbubS+pJKkBjzPTWpDRLyA6moJxwH/BTycmb/pN/9jVHdzeEMHtnUA8F1gj8x8qN31SSWz5ya15yhgZWZelpnrBtCXuYsAAAI0SURBVATb7wFvokNX0c/MW6iuqPGXnVifVDLDTWpRRNwOfBSYHdWd0i8bUORIqjsEfL/fMsdFxOMRsVu/aZ+MiJ82eXuSr1NdD1HSMAw3qXV/CNwO/D3VDRgXDZh/MHBjPnnf/6VU90b7e4CI+FuqsFqYmfc2sc0fAQdFxLZt1l0q2qje8kYqzK+pLgD9/cy8Z5D5uwHr+k/IzIyI9wL/HhE/pbrX1YLMvKPJbd5NdXfs51JdKFzSIOy5Sa2bRfUFcfkQ87cFHhs4MTO/A9xAdcPPP8vMG0awzUf7rVvSEAw3qXVzgJ9n5oNDzF8PTB44MSIWALOp7jz8lF2REXFFRHwwIn4QEWsiYm6/2VPqn/e3V3WpbIab1Lo5DN1rA/hvYL/+E+o7P38VeAfwNeD0QZabBazNzJcB/wQcM2De3U0en5Oetgw3qXWNwu3bwL4RMRWgHiH5TeCMzDwXeD9wWEQc0rdARGwPbJOZfSeDTwA29FvnwcCVHWuBVCjDTWpBRATwQoYJt/q8tB8Bx0fEFKpQ+kZmfrCev5LqLuT9e2+zqI7H9TmA6iRxIqKHqhd3VudaIpXJK5RIXRQRC4FPAvtl5uYmyp8CPCczP1Q//y/gNZl5b0S8vf798K5WWiqAPTepizLzSuBTwIwmFzkAuBme6B3u1O/42kaqY3WSGrDnJkkqjj03SVJxDDdJUnEMN0lScQw3SVJxDDdJUnEMN0lScQw3SVJxDDdJUnEMN0lScf4/MOVwERlmCvkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize = (7,5))\n",
    "sns.distplot(model.f[model.y == 0], bins = 25, kde = False, \n",
    "             color = 'cornflowerblue', label = 'Class 0')\n",
    "sns.distplot(model.f[model.y == 1], bins = 25, kde = False, \n",
    "             color = 'darkblue', label = 'Class 1')\n",
    "ax.set_xlabel(r\"$f\\hspace{.25}(x_n)$\", size = 14)\n",
    "ax.set_title(r\"Histogram of $f\\hspace{.25}(x_n)$ by Class\", size = 16)\n",
    "ax.legend()\n",
    "sns.despine()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
